Individual feature selection in each One-versus-One classifier improves multi-class SVM performance

Multiclass One-versus-One (OvO) SVM, which is constructed by assembling a group of binary classifiers, is usually treated as a black-box. The usual Multiclass Feature Selection (MFS) algorithm chooses an identical subset of features for every OvO SVM. We question whether the standard process of applying feature selection and then constructing the multiclass classifier is best. We propose that Individual Feature Selection (IFS) can be directly applied to each binary OvO SVM. More specifically, the proposed method selects different subsets of features for each OvO SVM inside the multiclass classifier so that each vote is optimised to discriminate between the two specific classes. This paper shows that this small change to the normal multiclass SVM improves performance and can also reduce the computing time of feature selection. The proposed IFS method is tested on four different datasets for comparing the performance and time cost. Experimental results demonstrate significant improvements compared to the normal MFS method on all four datasets.